1. Field of the Invention
This invention generally relates to a panoramic image reproduction system, and in particular to the use of rotary scanning elements in a camera design, with multiple sensors being used to establish stereoscopic viewpoints, adjust their convergence, and to increase the density of information being scanned.
2. Description of the Prior Art
Cameras for capturing still or motion pictures of the external world have usually used an area array sensor, such as a 640×480 Charge-Coupled Device (CCD) for NTSC video images. These area sensors use a rectangular or square area of light-sensitive elements and expose them all at once. New forms of Complementary Metal Oxide Semiconductor (CMOS) sensors for cameras are also being made in a variety of resolutions.
Linear scan sensors, either CMOS or CCD, are also being used to increase resolution for specific applications. These sensors have a single row of sensors, designed for monochrome or color readout, and sensitive to either visible light or some other part of the spectrum such as infrared or ultraviolet. The image is made by successive readouts of the sensor as it moves across the image, building up the final resolution. This is also sometime referred to as a slit scan or line scan sensor. Digital scans of this type are already found in such devices as fax machines and flatbed scanners. Line scan cameras are often used in industrial processes to obtain clear pictures of objects in motion. Examples of these cameras are the Wintriss Engineering OPSIS 5150ALC for monochrome images, which uses a monochrome CCD linear sensor with 5150 pixels and onboard image processing, and the DALSA CT-E4-2048W, which uses a color CCD linear sensor with 2048 pixels. Typically these cameras are fixed in place, and look at objects moving past them. Unlike area-scan sensors, the resulting images do not suffer from motion blur.
Scan-back digital cameras, such as the Better Light Model 8000 camera back are also doing high-quality digital still photography. The use of a slow scan from a linear sensor allows for maximum resolution to be built up when photographing artwork, documents or other subjects.
Scanning sensors are also used in so-called “push-broom” satellite surveillance applications, where as the satellite sweeps across the surface, a succession of readouts of the scanner creates a very large and detailed overall image. Existing panoramic slit scan cameras have also been made that use a moving slit to expose a strip of film, such as the Widelux™ camera for stills.
Various patents describe scanning cameras. All typically feature a rotation of the scanner to create the image according to an optical axis radial to the axis of rotation. Keller's “Panoramic Camera” (U.S. Pat. No. 5,659,804), for example, describes rotation of a scanner where the nodal point of the optics for the scanner is located radially according to the axis of rotation. Oxaal's “Method and Apparatus for Producing a 360° Spherical Visual Data Set” (U.S. Pat. No. 5,903,782) also involves a radial optical axis, this time for a “fisheye” lens. This is the approach used in the high-resolution digital PanoScan™ still camera. Jehle's “360 degree IR Surveillance with Panoramic Display” (U.S. Pat. No. 4,977,323) uses an infrared scanner also aligned radially to the axis of rotation. Federau's “Line-Scan Panoramic Camera” (U.S. Pat. No. 4,532,544) also describes a radial axis for a television camera. And Globus et al's “Panoramic Camera” (U.S. Pat. No. 4,241,985), marketed as the Globuscope™, employs a slit and a radial optical axis to expose a film strip.
Schonherr (U.S. Pat. No. 5,305,035) describes a design for slots that are intended to “seal off” the edges of a light-sensitive cylindrical recording surface when used with a rotating objective drum, thus preventing extraneous light at the edges of the recording, or “light safety”. The drum of Schoenherr et al has a lens in its center which focuses light from a wide input image window on one side onto a narrow image output window on the other side, so that as the drum rotates, the light passes through this second window in a scanning fashion onto the fixed cylindrical recording surface outside of the drum. Because the recording surface is separate and outside of the drum, this device has a horizontal field of view that is inherently less than 180 degrees. Other film strip-based scanning cameras have also been made, going back to the cumbersome Cirkut camera of the nineteenth century. Contemporary examples include the Widelux, which exposes a film strip in 120-degree increments of rotation, and the Hulcherama camera which exposes a continuous strip of film in any amount of rotation, using gear motors to advance the film in time with the sweep of the lens. Woltz's “Panoramic Motion Picture Camera and Method” (U.S. Pat. No. 4,602,857) describes a rotating motion picture film camera. All of these cameras share the same orientation of the optical axis as being radial to the axis of rotation.
Keast, et al (U.S. Pat. No. 5,721,585) discloses a rotating camera which in one embodiment has two cameras with axes not radial to the axis of rotation. However, Keast depends upon the addition of counter-rotation relay optics between the lens and the sensor. This adds significant weight and complexity to the device, and also requires that a perfectly consistent geometrical relationship be consistently maintained between the input lens and the sensor along a long optical path, even while the whole apparatus is in rapid motion and presumably being carried in the field in the manner of a regular camera. The described counter-rotation optics are also inherently impractical, because as the overall apparatus rotates, at the point where the lens is opposite the sensor, the mirror surfaces and their optical supports need to completely disappear in order to avoid an interruption in the image.
What these solutions lack is an effective way to record not only the appearance of the real world in motion through the slit, but also to use that information to create a stereoscopic image of the entire field of view. Without that stereoscopic information, the most realistic picture of a surrounding scene is not possible, as well as other applications such as rangefinding that depend on parallax offsets between different points of view. The reason for this is that existing rotary solutions assume an optical axis for the light path that is radial to the axis of the rotating array. This restriction precludes the use of other axes in a rotating array for stereoscopic effects or increased resolution. If one attempted to create a stereoscopic view with two panoramic scans taken with such cameras, by taking them from two laterally offset points in space, the stereo illusion would be strong only for the area directly orthogonal to the axis connecting the two points, but would fall off to zero as you approached that axis in the course of looking around. This is described in McMillan & Bishop's “Plenoptic Modelling: An image-Based Rendering System” in SIGGRAPH 95 Conference Proceedings pp. 39-46. An overview of stereoscopic imaging systems, including relevant US and foreign patents, can be found in Michael Starks' summary article “Stereoscopic Imaging Technology” from 3DTV Corporation (http://www.3dmagic.com).
In a pair of stereoscopic panoramas, the lateral offsets between the objects in them represent their distances from the camera. However, these offsets can be dynamically removed by digital or other means, on a localized basis, to produce a more continuous image, with the amounts of correction also reflecting a depth map of the scene. This dynamic correction is in contrast with the static approaches of the prior art. Sheiman et al (U.S. Pat. No. 4,235,515), for example, discloses a fixed, physical viewing method using prisms in sheets to establish apparent dual images for stereo viewing, and is directed at creating a fixed setting that removes the overall lateral distortion between the images.
The use of multiple cameras can not only establish two stereoscopic viewpoints, but also be used to increase resolution and light sensitivity in the recording process. Keast et al describes the use of two cameras, purely for establishing stereo viewpoints, and the counter-rotation relay optics described preclude more than two cameras from being used. In addition, the described method suggests that there would be significant light interference between the light from the two cameras.
The present invention will address all of these shortcomings in the prior art. The present application is based in part on the disclosure document no. 466771 “Rotating Scan Camera with Oblique Axis” filed Dec. 22, 1999, and is a continuation-in-part of application Ser. No. 09/505,601 “Rotating Scan Camera” filed Feb. 17, 2000.